This blog post introduces the Semantic Telemetry project at Microsoft Research, which uses a data science approach to analyze how people interact with AI systems, specifically focusing on Copilot in Bing usage. It discusses the complexity of human-AI interactions and how they differ from traditional search.
- Topics: Copilot in Bing chats were analyzed for topic categorization. Technology (21%) was the most common topic, followed by Entertainment (12.8%), Health (11%), and others. Within technology, programming and scripting were prominent subtopics.
- Platform Differences: Mobile users tend to use Copilot for personal tasks, while desktop users engage in more professional activities.
This paper proposes a new method called MoRA for parameter-efficient fine-tuning of large language models (LLMs). The proposed method, MoRA, employs a square matrix to achieve high-rank updating, maintaining the same number of trainable parameters. The paper suggests that low-rank updating, as implemented in LoRA, may limit the ability of LLMs to effectively learn and memorize new knowledge. MoRA outperforms LoRA on memory-intensive tasks and achieves comparable performance on other tasks.
